1. Field of the Invention
The present invention generally relates to image acquisition and processing, and, in particular, to a system and method that integrates spatial image resolution scaling, commonly called image resizing, on an image sensor.
2. Related Art
A digital camera typically employs image sensors to convert light reflected by an object through an optical lens into electrical signals. Sensor technology utilized in digital cameras may employ a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) image sensor to convert the light (photons) into electrical signals (electrons).
The sensor technology in a digital camera typically includes an array of photodiodes, referred to in the art as “photosites.” The array of photosites is conceptually a two-dimensional array, and each photosite in the array converts a small portion of the total light detected, which is reflected from an object within the lens' field of view, into electrical signals.
Typically, the electrical signals are analog signals, which are converted into digital format by an analog-to-digital (A/D) converter. The A/D converter may either be located on the sensor (e.g., on a portion of the semiconductor chip, which comprises a CMOS image sensor) or the A/D converter may receive the output of the sensor. In either embodiment, the A/D converter typically converts the analog signal to a digital signal, and then transmits the digital signal to a general-purpose processor or application specific integrated circuit (ASIC) within the digital camera.
The processor of the digital camera performs a wide variety of image processing functions. These operations commonly include auto-exposure, auto-focus, automatic white balancing, sharpening, image spatial resolution scaling, compression as well as other proprietary image processing operations known in the art. After the digital image data is fully processed, it is usually stored in digital format in memory located within the digital camera.
Recent technological advances have resulted in sensors for digital cameras, which can produce outputs in the several megapixels. For example, a typical CMOS image sensor or CCD sensor can employ a 2000×1500 array of photosites, which output three million analog signals. If three million analog signals representative of light detected by the photosites are digitized, then a typical representation of one single still image or frame would comprise approximately three million image data values. Moreover, if the digital camera were in video mode to capture motion, then a full photosite array collection may be three million pixels at up to thirty frames per second. Common frame rates for full motion digital video range between 24 and 30 frames per second.
Even if the sensors are equipped to collect megapixels of data at video frame rates, typically, the bus bandwidth and data processing capabilities of digital cameras are unable to manipulate such a large volume of data. In other words, the computation requirements necessitated by such a large volume of data (e.g., three million pixels per frame at a rate of 30 Hz) would easily exceed the processing capabilities of the system.
To reduce the volume of image data generated by the image sensor and, therefore, consumed by the camera's processor, sensors may include logic that performs a process known in the art as “pixel binning.” Generally, pixel binning refers to collecting multiple pixel signals in the vertical and/or horizontal direction, into a single larger charge, which represents the area of the individual pixels contributing to the charge. In this regard, as light falls on an image sensor, for example a CCD, electrons accumulate at each pixel. Note that the number of electrons that a pixel is capable of accumulating varies, depending upon the type of image sensor employed. However, for exemplary purposes, assume that each pixel of the image sensor is capable of accumulating 100,000 electrons at each pixel, which is commonly referred to in the art as “well depth.” A group of the pixels are then “binned” by summing the accumulated electrons at each pixel in the group into a single charge, or “superpixel.”
For example, a binning of 3×3 means that an area of nine (9) adjacent pixels have been combined into one large pixel. Employing the example provided hereinabove, if each pixel is capable of accumulating 100,000 electrons, and each pixel in the nine adjacent pixels accumulate 80,000 electrons, then the superpixel that results from binning the nine pixels with a 3×3 matrix is a single “superpixel” comprised of 720,000 electrons.
Moreover, the 3×3 binning applied to a sensor that has a photosite array of 2000×1500, results in data collected that comprises a two-dimensional array of 667×500. Therefore, the amount of collected data for each frame is reduced in size from three megapixels to approximately three-hundred thousand (300,000) pixels. This reduction in the sensor's output data rate is intended to bring the sensor's output data rate into the processing capability of the camera's processor.
Note that the “superpixel” method of binning electrons from an image sensor is an example of a binning method known in the art. Other binning methods known in the art may also be implemented to reduce the volume of image data produced by the image sensor.
Typically, after electrical signals are binned, the data is transmitted from the sensor to a general-purpose processor or an ASIC for video processing operations.
Image resizing is generally performed by a processor or an ASIC, so that the output image size corresponds to a desired resolution. For example, the digital camera may be set to capture an image that is 640 by 480 pixels, which may be significantly fewer pixels than the sensor is capable of generating. Another resolution commonly used in capturing video images is what is referred to as quarter VGA, which is a resolution of 320 by 240 pixels.
Moreover, a processor or ASIC that receives the digital data produced by an image sensor may reduce the image defined by the digital data to a desired resolution for more compact storage or the processor or ASIC may not perform any resizing of the digital data, prior to its storage.